Diagnostic and antibiotic stewardship lessons: an outpatient assessment of symptomatic reflex urinalysis ordering accuracy using an electronic best-practice alert.

BACKGROUND: It is not well known how reliably clinicians order reflex urinalysis to microscopy and culture (rUA-cx) for outpatient urinary tract infection (UTI) workup. Antibiotic appropriateness cannot be fully appreciated until the prevalence of UTIs and asymptomatic bacteriuria (ASB) are realized. OBJECTIVE: This quality improvement study has two major aims, first to determine UTI symptom accuracy for rUA-cx ordering and second, to confirm UTI and ASB cases by integrating rUA-cx and cascaded urinalysis results. Antibiotic utilization and diagnostic coding were secondarily linked to UTIs and ASB. METHODS: An electronic best-practice alert informed the ordering of two rUA-cx options: symptomatic- rUA-cx specifically for dysuria, frequency, urgency, costovertebral pain, suprapubic pain or fever versus non-specific-rUA-cx for vague complaints. UTI symptoms were verified by chart review. Confirmed UTI was defined as a significant culture with UTI symptoms and ASB as a significant culture without UTI symptoms. RESULTS: rUA-cx (2065) were prospectively collected over 6 months from female patients at risk for uncomplicated UTIs. Symptomatic-rUA-cx and non-specific-rUA-cx were associated with UTI symptoms for 53% (809/1527) and 20% (107/538), respectively. Overall, 44% (916/2065) of all rUA-cx had UTI symptoms. rUA-cx were overordered by a factor of 9 (2065/225) for every confirmed UTI. The UTI-to-ASB relative ratio was 2.6 (225/86). Regarding UTI-relevant antibiotics, 39% (214/553) were appropriately associated with UTI whereas only 22% (74/339) of inappropriate antibiotics were captured by the ASB definition, underestimating the problem 4-fold. CONCLUSIONS: UTI and ASB remain challenging to categorize despite a meticulous method that applied acceptable criteria.

Improving Methods of Identifying Anaphylaxis for Medical Product Safety Surveillance Using Natural Language Processing and Machine Learning.

We sought to determine whether machine learning and natural language processing (NLP) applied to electronic medical records could improve performance of automated health-care claims-based algorithms to identify anaphylaxis events using data on 516 patients with outpatient, emergency department, or inpatient anaphylaxis diagnosis codes during 2015-2019 in 2 integrated health-care institutions in the Northwest United States. We used one site's manually reviewed gold-standard outcomes data for model development and the other's for external validation based on cross-validated area under the receiver operating characteristic curve (AUC), positive predictive value (PPV), and sensitivity. In the development site 154 (64%) of 239 potential events met adjudication criteria for anaphylaxis compared with 180 (65%) of 277 in the validation site. Logistic regression models using only structured claims data achieved a cross-validated AUC of 0.58 (95% CI: 0.54, 0.63). Machine learning improved cross-validated AUC to 0.62 (0.58, 0.66); incorporating NLP-derived covariates further increased cross-validated AUCs to 0.70 (0.66, 0.75) in development and 0.67 (0.63, 0.71) in external validation data. A classification threshold with cross-validated PPV of 79% and cross-validated sensitivity of 66% in development data had cross-validated PPV of 78% and cross-validated sensitivity of 56% in external data. Machine learning and NLP-derived data improved identification of validated anaphylaxis events.

Validation of Acute Pancreatitis Among Adults in an Integrated Healthcare System.

BACKGROUND: Acute pancreatitis is a serious gastrointestinal disease that is an important target for drug safety surveillance. Little is known about the accuracy of ICD-10 codes for acute pancreatitis in the United States, or their performance in specific clinical settings. We conducted a validation study to assess the accuracy of acute pancreatitis ICD-10 diagnosis codes in inpatient, emergency department (ED), and outpatient settings. METHODS: We reviewed electronic medical records for encounters with acute pancreatitis diagnosis codes in an integrated healthcare system from October 2015 to December 2019. Trained abstractors and physician adjudicators determined whether events met criteria for acute pancreatitis. RESULTS: Out of 1,844 eligible events, we randomly sampled 300 for review. Across all clinical settings, 182 events met validation criteria for an overall positive predictive value (PPV) of 61% (95% confidence intervals [CI] = 55, 66). The PPV was 87% (95% CI = 79, 92%) for inpatient codes, but only 45% for ED (95% CI = 35, 54%) and outpatient (95% CI = 34, 55%) codes. ED and outpatient encounters accounted for 43% of validated events. Acute pancreatitis codes from any encounter type with lipase >3 times the upper limit of normal had a PPV of 92% (95% CI = 86, 95%) and identified 85% of validated events (95% CI = 79, 89%), while codes with lipase <3 times the upper limit of normal had a PPV of only 22% (95% CI = 16, 30%). CONCLUSIONS: These results suggest that ICD-10 codes accurately identified acute pancreatitis in the inpatient setting, but not in the ED and outpatient settings. Laboratory data substantially improved algorithm performance.

Decrease in unnecessary vitamin D testing using clinical decision support tools: making it harder to do the wrong thing.

OBJECTIVE: To evaluate the impact of clinical decision support (CDS) tools on rates of vitamin D testing. Screening for vitamin D deficiency has increased in recent years, spurred by studies suggesting vitamin D's clinical benefits. Such screening, however, is often unsupported by evidence and can incur unnecessary costs. MATERIALS AND METHODS: We evaluated how rates of vitamin D screening changed after we implemented 3 CDS tools in the electronic health record (EHR) of a large health plan: (1) a new vitamin D screening guideline, (2) an alert that requires clinician acknowledgement of current guidelines to continue ordering the test (a "hard stop"), and (3) a modification of laboratory ordering preference lists that eliminates shortcuts. We assessed rates of overall vitamin D screening and appropriate vitamin D screening 6 months pre- and post-intervention. RESULTS: Vitamin D screening rates decreased from 74.0 tests to 24.2 tests per 1000 members ( P < .0001). The proportion of appropriate vitamin D screening tests increased from 56.2% to 69.7% ( P < .0001), and the proportion of inappropriate screening tests decreased from 43.8% pre-implementation to 30.3% post-implementation ( P < .0001). DISCUSSION: To our knowledge, this is the first demonstration of how CDS can reduce rates of inappropriate vitamin D screening. We used 3 straightforward, inexpensive, and replicable CDS approaches. We know of no previous research on the impact of removing options from a preference list. CONCLUSION: Similar approaches could be used to reduce unnecessary care and decrease costs without reducing quality of care.

Incidence and risk factors for venous thromboembolic disease in podiatric surgery.

BACKGROUND: The Agency for Healthcare Research and Quality ranks prevention of venous thromboembolism (VTE) as a top priority for patient safety; however, no guidelines or population-based research exist to guide management for podiatric surgery patients. The objective of our study was to determine the incidence and risk factors for postprocedure VTE in podiatric surgery. METHODS: A 5-year retrospective analysis of patients undergoing podiatric surgery in a large not-for-profit health maintenance organization serving > 485,000 members in the Pacific Northwest from 1999 to 2004. RESULTS: We identified 16,804 surgical procedures in 7,264 patients and detected 22 symptomatic postprocedure VTEs. The overall incidence of postprocedure VTE was 0.30%. Three risk factors were significantly and independently associated with VTE in podiatric surgery: prior VTE (incidence, 4.6%; relative risk, 23.0; p < 0.001), use of hormone replacement therapy or oral contraceptives (incidence, 0.55%; relative risk, 4.2; p = 0.01), and obesity (incidence, 0.48%; relative risk, 3.0; p = 0.02). CONCLUSIONS: We identified a low overall risk of VTE in podiatric surgery, suggesting that routine prophylaxis is not warranted. However, for patients with a history of VTE, periprocedure prophylaxis is suggested based on the level of risk. For podiatry surgery patients with two or more risk factors for VTE, periprocedure prophylaxis should be considered. Until a prospective study is completed testing recommendations, guidelines and care decisions for podiatric surgery patients will continue to be based on retrospective data, expert consensus, and clinical judgment.